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Abstract 

Objective To determine whether use of the Professional and Linguistic 
Assessments Board (PLAB) examination system used to grant 
registration for international medical graduates results in equivalent 
postgraduate medical performance, as evaluated at Annual Review of 
Competence Progression (ARCP), between UK based doctors who 
qualified overseas and those who obtained their primary medical 
qualification from UK universities. 

Design Observational study linking ARCP outcome data from the UK 
deaneries with PLAB test performance and demographic data held by 
the UK General Medical Council (GMC). 

Setting Doctors in postgraduate training for a medical specialty or 
general practice in the UK and doctors obtaining GMC registration via 
the PLAB system. 

Participants 53 436 UK based trainee doctors with at least one 
competency related ARCP outcome reported during the study period, 
of whom 42 017 were UK medical graduates and 1 1 419 were 
international medical graduates who were registered following a pass 
from the PLAB route. 

Main outcome measure Probability of obtaining a poorer versus a more 
satisfactory category of outcome at ARCP following successful 
registration as a doctor in the UK. 

Results International medical graduates were more likely to obtain a 
less satisfactory outcome at ARCP compared with UK graduates. This 
finding persisted even after adjustment for the potential influence of sex, 
age, years of UK based practice, and ethnicity and exclusion of outcomes 
associated with postgraduate examination failure (odds ratio 1 .63, 95% 
confidence interval 1 .30 to 2.06). However, international medical 
graduates who scored in the highest twelfth at part 1 of the PLAB (at 



least 32 points above the pass mark) had ARCP outcomes that did not 
differ significantly from those of UK graduates. 

Conclusions These findings suggest that the PLAB test used for 
registration of international medical graduates is not generally equivalent 
to the requirements for UK graduates. The differences in postgraduate 
performance, as captured at ARCP, following the two routes to 
registration might be levelled out by raising the standards of English 
language competency required as well as the pass marks for the two 
parts of the PLAB test. An alternative might be to introduce a different 
testing system. 

Introduction 

The healthcare workforce is becoming increasingly international 
and globalised.' In the United Kingdom in 2012, 37% of the 
doctors registered with the national medical regulator, the 
General Medical Council (GMC), had qualified in other 
countries; 27% had obtained their medical degree from outside 
the European Economic Area (EEA). 2 Before their names are 
placed on the GMC's List of Registered Medical Practitioners, 
allowing the doctor to legally practise medicine in the UK, all 
doctors must fulfil the criteria laid out in the 1983 Medical Act. 3 
For international medical graduates, several routes to such 
registration exist (box 1). 

To take the Professional and Linguistic Assessments Board 
(PLAB) test (box 2), international medical graduates must 
provide evidence of language competency in English. This 
usually means having passed an International English Language 
Test System (IELTS) assessment with a score of band seven 
(good user) or above, before applying for PLAB part 1. The 
IELTS test is in four parts — listening, speaking, reading, and 
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Box 1 Requirements for GMC registration for international medical graduates 

Most international medical graduates register with the GMC via the Professional and Linguistic Assessments Board (PLAB) test 

To sit the PLAB, candidates must hold a medical degree from a recognised institution, 5 provide evidence of competency in English, and 
have undergone at least 12 months of postgraduate clinical experience 

If an approved UK health provider offers sponsorship to an overseas doctor with at least three years' postgraduate training the GMC 
can issue a certificate of sponsorship, 6 allowing the doctor to practise for a limited time 

International medical graduates may apply for a Certificate of Eligibility for Specialist Registration (CESR), 7 which would allow them to 
practise in the UK at consultant level. This involves submitting a portfolio of evidence of competence to the relevant medical royal college 
for evaluation 

Similarly, overseas doctors can apply for a Certificate of Eligibility for General Practice Registration (CEGPR), 8 allowing them to work 
as general practitioners in the UK. This involves providing a portfolio of evidence to the Royal College of General Practitioners for 
evaluation, as well as obtaining two structured referees' reports 



writing — and can be taken as many times as desired. 4 Each part 
is graded between band one (non-user) and band nine (expert 
user). 

The PLAB test is designed to ensure that overseas doctors 
demonstrate clinical competencies that would be equivalent to 
those that a UK graduate would be expected to have achieved 
by the end of foundation year one. This is a basic initial training 
year immediately following medical graduation, which consists 
of clinical placements focused on medicine and surgery, 
although attachments in other specialties such as psychiatry and 
emergency medicine are increasingly being offered. Once 
foundation year one has been completed, foundation year two 
posts can be applied for; these continue basic clinical training, 
although with an expectation of additional autonomy and 
responsibilities. Thus, a successful PLAB graduate is able to 
apply for foundation year two posts, in competition with both 
UK and EEA medical graduates. 

All doctors in UK postgraduate training are assessed regularly. 
Trainees who started before 1 August 2007 are covered by the 
Record of In-Training Assessment (RITA).' Trainees who started 
on or after 1 August 2007 are assessed by an Annual Review 
of Competence Progression (ARCP). 1 " 

An ARCP (or previously RITA) panel is composed of at least 
three senior doctors working in the relevant specialty. One 
member must be a postgraduate dean or training programme 
director, and at least two must be drawn from a list that includes 
a royal college representative, an educational supervisor, and 
other holders of roles related to postgraduate medical training. 
There should also be a representative of the doctor's employer. 10 
The assessment is generally conducted annually but may be 
carried out more frequently if there are concerns or other 
circumstances, such as insufficient evidence presented, that 
require this. The panel considers evidence, presented by the 
trainee and his or her trainer at that time, of the trainee's 
progression. The panel then decides whether the trainee is 
competent in the required skills and has the necessary knowledge 
to progress to the next level of training. Such evidence may 
include passing a postgraduate examination required for 
membership of the relevant royal college. The royal colleges 
set the curriculum and the competencies that would be expected 
to be achieved at each level of training in their specialty. The 
timing of these examinations varies across specialties, but in 
most instances one would be expected to undergo a defined 
period of approved UK postgraduate training before being 
permitted to sit an examination. There are some exceptions — for 
instance, one can sit the Membership of the Royal College of 
Physicians (MRCP) examination, part 1, after 12 months of 
non-UK medical practice. Failure to pass royal college 
examinations can sometimes, but not invariably, impede 
progress in training, depending on the specialty and examination. 
For example, to enter into higher specialist training in psychiatry 
(the last three years of postgraduate training leading to eligibility 



for employment at consultant level), one must have passed all 
three parts of the relevant membership examination. At ARCP, 
the trainee would present a portfolio of structured observed 
tasks and procedures that have been assessed by the trainer and 
other practitioners involved in the doctor's training. 

Table 1 U outlines the possible outcomes at ARCP and RITA. 13 
If a doctor has not developed a specific competency, but is 
generally deemed to be progressing adequately, the panel may 
recommend that some targeted training is provided without an 
extension to the training period. If more serious deficiencies in 
competencies are perceived, the panel may recommend that the 
usual training period is extended. Following this, if satisfactory 
progress is still not made, the doctor may have to leave the 
training programme. "Out of programme" experience does not 
include periods of maternity leave. 

Recently, there has been a significant focus on the clinical 
performance of overseas doctors. A study reported that, 
compared with white UK graduates, non-white international 
medical graduates were almost 15 times more likely to fail the 
clinical skills assessment component of the Membership of the 
Royal College of General Practitioners (MRCGP) examination, 
even after adjustment for sex, age, and performance on the 
applied knowledge test of the assessment. 14 In addition to 
concerns relating to postgraduate examination performance, 
evidence suggests that, compared with UK graduates, 
international medical graduates are more likely to be referred 
or censured for concerns relating to fitness to practise, even 
after control for the potential influence of enquiry characteristics 
(source, type, and content), sex, time since qualification, and 
specialty (general practice versus non-general practice). 15 

The GMC commissioned the analysis reported here as part of 
a review of the PLAB system, and it builds on a recent review 
of the test. 16 17 A key focus was "to examine whether 
international medical graduates granted full registration 
following ... the PLAB test are more or less likely than other 
. . . doctors to experience difficulties in medical practice." This 
study was intended to complement a separate strand of research 
examining differential pass rates at the MRCP and MRCGP 
examination. 18 However, by using ARCP rating as the outcome 
of interest, we were able to evaluate the performance of doctors 
in postgraduate training across all medical specialties, not just 
medicine and general practice for which the MRCP and MRCGP 
examinations are respectively taken. Moreover, assessment at 
ARCP takes into account a wider range of factors than 
postgraduate examination passes. We anticipated that the results 
would shed light on the magnitude of any existing differences 
in performance and elicit some of the potential mediating and 
moderating influences for any disparities observed. We hoped 
that the findings would also provide indications of whether the 
PLAB test could be modified to increase its effectiveness in 
encouraging equivalence between international medical 
graduates and UK graduates. 
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Box 2 Professional and Linguistic Assessments Board (PLAB) test 

The PLAB test is in two parts, which must be passed sequentially and may be retaken as many times as desired 

Part 1 

Part 1 of the PLAB test assesses the medical knowledge of candidates as relevant to the UK. It is a three hour examination with 200 
multiple choice questions for which the candidate has to select the single best answer. The test covers the following areas: applying 
knowledge and experience to practise, clinical care, assessment, and clinical management 

The pass mark is set by a variant of the Angoff method, whereby experts decide the minimum scores that would be acceptable for the 
test items" 



Part 2 

Part 2 of the test must be passed within three years of passing part 1 

It consists of 14 objective structured clinical examination (OSCE) stations. Each station consists of a five minute clinical scenario in 
which candidates are observed by a single examiner and scored on their use of skills and knowledge 

The skills assessed are clinical examination, practical skills, communication skills, and history taking 

The pass mark for part 2 is set via the borderline group scoring method. 12 This involves weighting the scores for the OSCE stations. 
Weightings are decided according to expert opinion of the score expected of a "minimally competent" candidate and are informed by 
scores obtained by previous cohorts 



Methods 

Our aims were to compare ARCP outcomes between 
international medical graduates who had passed the PLAB and 
UK graduates, both before and after controlling for the effects 
of potential confounding factors, such as age, sex, and duration 
of UK based experience. The technical appendix gives a more 
detailed description of the data collection and classification, 
model building, missing data patterns, and data imputation, 
along with a number of supplementary results. 

Data sources 

Data on ARCP outcomes from the UK deaneries (administrative 
regions responsible for postgraduate medical training at the 
time 19 20 ) for 2010-12 came from the GMC. We converted 
outcomes relating to the RITA into the equivalent ARCP 
outcomes where applicable. In total, 125 208 ARCP outcomes 
relating to educational progress and performance were available 
for 53 436 doctors who were either UK graduates or 
international medical graduates who had passed the PLAB test. 
Figure 1|| shows the flow of study data. The IELTS scores 
(graded 7.0 to 9.0) were available for 25 896 of international 
medical PLAB graduates. PLAB test data were available for all 
27 726 international medical graduate candidates who passed 
the examination from July 2000 to the end of 2010. As pass 
marks vary between diets (sittings), we converted the raw PLAB 
scores to the scores relative to the pass mark for that sitting. 
Multiple resits of postgraduate medical examinations may be 
associated with disproportionately poor performance. 21 
Therefore, we explored the potential association between the 
number of PLAB sittings and later ARCP performance. 

The research team had access to a dichotomised version of 
ethnicity (white/non-white) obtained via the GMC National 
Training Survey. Additional demographic data such as sex, 
world region where the primary medical qualification was 
obtained, date of birth, and date of first registration (or 
provisional registration if present) came from the List of 
Registered Medical Practitioners. 22 

Model building 

For the purposes of our analyses, we used random effects ordinal 
logistic regression to estimate the odds of receiving a less 
satisfactory (rather than a more satisfactory) ARCP outcome 
depending on the world region of qualification (international 
medical graduate versus UK graduate). For international medical 
graduates, we were also able to assess how IELTS and PLAB 
scores were associated with ARCP outcomes. We also conducted 



analyses controlling for the effects of several potential 
confounding variables such as age, sex, IELTS, years of UK 
based experience, and ethnicity. Individual doctors had each 
undertaken varying numbers of ARCPs during the study period. 
Therefore, we used a multilevel model whereby ARCP outcomes 
were conceptualised as nested within individual doctors. 

We conducted "simulation" of a change in pass mark by dividing 
PLAB graduates into twelfths on the basis of their examination 
performances at pass. We used twelfths rather than tenths to 
provide a slightly finer grained picture. We then compared these 
subgroups of PLAB graduates with UK graduates. 

Results 

Statistical significance should be assumed to be at the P<0.001 
level, unless otherwise stated. 

Descriptive statistics 

Table 2JJ shows the demographic characteristics of the doctors 
and degree of missing data. Compared with the UK graduates, 
international medical graduates were older and more likely to 
be male, to have had less time practising in the UK, to be of 
non-white ethnicity, and to have undergone more ARCPs in the 
study period. This could have indicated either longer training 
or a tendency to have more frequent assessments. As can be 
seen in table 2Jj, few data were missing with the exception of 
ethnicity, which was missing in just under a quarter of cases. 
The sample was drawn from all medical specialties. We grouped 
these into 12 specialties largely on the basis of the UK royal 
colleges associated with the training scheme. 23 

ARCP outcomes 

Figure 2|J depicts the proportion of the five main categories of 
ARCP outcome in the two groups of doctors. In total, 11 419 
international medical graduates who registered via the PLAB 
test underwent 28 189 ARCPs (excluding "out of programme 
experience"). We had data on 42 017 UK graduates undergoing 
97 019 ARCPs. As can be seen, international medical graduates 
are underrepresented compared with UK graduates in the 
categories representing adequate progression (that is, ARCP 
outcomes 1 and 6 and RITA C and G) and overrepresented in 
the other, less satisfactory, categories (for example, ARCP 
outcome 3: "extended training required"). Figure 31) shows the 
ARCP outcomes when those associated with reported 
examination failure are excluded. As expected, the proportion 
of less satisfactory ARCP outcomes falls for both groups, 
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although intergroup differences remain. We also observed that 
UK graduates were significantly more likely than international 
medical graduates to report having "out of programme 
experience." Of 54 122 doctors with any ARCP results recorded, 
only 530 (5%) of the 1 1 418 international medical PLAB 
graduates were recorded as having this as an ARCP outcome 
category. This contrasts with UK graduates, of whom 3582 (9%) 
of 42 017 doctors had this outcome recorded at least once. 

Univariable analysis 

For the purposes of the analyses, we treated ARCP outcome as 
an ordinal (ordered categorical) variable and coded it as follows: 
l=satisfactory progression; 2=insufficient evidence presented; 
3=targeted training required (but training time not extended); 
4=extended training time required/left programme. 

To estimate the unadjusted (raw) odds of obtaining a less 
satisfactory ARCP outcome according to the predictor variables 
(world region of qualification, IELTS and PLAB scores) we 
conducted a series of ordinal logistic regressions. International 
medical graduates who registered via the PLAB had more than 
twice the odds of obtaining a less satisfactory ARCP outcome 
compared with UK graduates (odds ratio 2.46, 95% confidence 
interval 2.35 to 2.58). Increasing age (odds ratio 1.04, 1.03 to 
1.04), male sex (1.42, 1.37 to 1.48), and non- white ethnicity 
(1.98, 1.88 to 2.08) were also associated with significantly 
increased odds of obtaining a less satisfactory outcome at ARCP. 
In contrast, UK based experience increased the odds of a more 
satisfactory ARCP rating by 1 0% per year (odds ratio 1.10, 1 .09 
to 1.11). (As in logistic regression, obtaining the odds ratios for 
the reverse outcome (in this case a more satisfactory ARCP 
outcome) is achieved by dividing 1 by the original odds ratios.) 

Table 3 ;; depicts the proportion of international medical 
graduates in the sample, according to medical specialty, along 
with the proportion of ARCPs that were reported as being 
satisfactory (that is, ARCP outcome 1 or 6, or the RITA 
equivalent). We also obtained unadjusted odds ratios for each 
medical specialty group, both with and without inclusion of 
ARCP outcomes associated with postgraduate examination 
failure. Figure 4.1.1 shows the results, which suggest that in most 
specialties international medical graduates have a higher odds 
of obtaining less satisfactory ARCP outcomes compared with 
UK graduates. Perhaps the two most striking features in figure 
4JL are the very high odds ratios of being in a less satisfactory 
ARCP category if you are an international medical graduate in 
general practice or psychiatry and the reduction in these 
disparities for these two specialities once ARCP outcomes 
related to examination failure are excluded. 

On univariate analysis, both IELTS and PLAB scores (relative 
to pass mark at first sitting) were significantly predictive of 
more satisfactory ARCP outcomes. For IELTS, each point above 
7.0 scored by the doctor being assessed increased the odds of a 
more satisfactory ARCP outcome by 69% (odds ratio 1 .69, 1 .55 
to 1.85). For PLAB part 1, each point gained above the pass 
mark increased the odds of being in a more satisfactory category 
by 2% (odds ratio 1.02, 1.02 to 1.02). For PLAB part 2, this 
value was approximately 8% (odds ratio 1.08, 1.08 to 1.09). 

We also "simulated" the effect of raising the entry level mark 
for both IELTS (the main route to PLAB) and the pass scores 
for the PLAB. Figure 5IJ depicts the unadjusted odds ratios for 
obtaining a less satisfactory ARCP outcome according to world 
region of qualification. As seen in figure 5 il, international 
medical graduates scoring the minimum 7.0 on IELTS required 
for PLAB examination entry are more likely to receive a less 
satisfactory ARCP outcome than are UK graduates. However, 



the disparity between international medical graduates and UK 
graduates decreases with increase in IELTS scores; PLAB 
graduates with IELTS scores of 9.0 have only a 50% higher 
odds of receiving a less satisfactory ARCP outcome compared 
with UK graduates. 

Figures 6JJ, and 7|l show that as the PLAB scores (at pass) for 
international medical graduates increase the discrepancy in 
performance at ARCP between this group and UK graduates 
decreases. This difference reaches statistically non-significant 
levels only for a PLAB part 1 score that would place a candidate 
in the top twelfth of PLAB graduates. This is equivalent to 
around 32 points above the current pass mark (that is, 1.8 
standard deviations above the mean for the present sample of 
PLAB candidates). According to figure 7 J, no realistic pass 
mark could be set for PLAB part 2 that would ensure overall 
equivalent ARCP performance between international medical 
graduates and UK graduates. 

Multivariable analysis 

To explore the extent to which PLAB status and performance 
were independently predictive of ARCP outcome, we developed 
a series of ordinal logistic regression models controlling for the 
potential effects of demographic variables. Missing values for 
ethnicity were multiply imputed. 24 For these analyses, we 
excluded ARCP outcomes associated with examination failure. 
By doing so, we hoped to elicit some of the mediators between 
region of qualification and postgraduate performance beyond 
those related to examination pass rates. 

As can be seen in table 4 J , even after we controlled for the 
effects of sex, mean age at ARCP, and years of UK based 
experience, and allowing for significant interactions, 
international medical graduates are more likely to receive a less 
satisfactory ARCP outcome than are UK graduates. Age 
interacted with UK based experience, indicating that in older 
doctors UK based experience was less strongly predictive of a 
more satisfactory ARCP outcome. A significant interaction also 
existed between PLAB graduate status and ethnicity, which 
suggested that being of non- white ethnicity was not as predictive 
of a less satisfactory ARCP outcome in international medical 
graduates, compared with UK graduates. Lastly, an interaction 
existed between age and ethnicity, which suggested that in older 
doctors the association between non-white ethnicity and 
obtaining a less satisfactory ARCP outcome was accentuated. 

PLAB performance 

PLAB scores at both parts of the examination (at first attempt) 
significantly predicted ARCP performance, even after we 
controlled for the potential effects of demographic factors and 
IELTS scores. Ethnicity was not included as a covariate in these 
models, as it was not a significant predictor of ARCP outcome 
in international medical graduates according to the results of a 
univariate analysis. Table 5IJ shows the results. We noted one 
significant interaction between the score obtained at PLAB part 
1 and years of UK based experience. This suggests that 
candidates with higher scores at PLAB part 1 and who gained 
more UK based clinical experience had a particular advantage 
when later undergoing an ARCP. As can be seen in table 5 J, 
PLAB part 1 score had no effect on ARCP outcome except 
through the interaction with UK based experience. We may 
therefore speculate that the benefits of obtaining a higher PLAB 
part 1 score, which is knowledge based, may generally only be 
realised at ARCP in the presence of appropriate clinical exposure 
and training. 
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We also explored the effect of re-sitting the PLAB on future 
ARCP performance. As can be seen in table 6 II, compared with 
passing the PLAB parts after multiple attempts, passing at first 
sittings was independently and significantly predictive of a more 
satisfactory outcome at ARCP. For PLAB part 1, this effect 
seems to level off at three attempts (that is, two resits), whereas 
for part 2 these effects level off at two attempts (one resit). 

Discussion 

This is the first study to report on overall postgraduate 
performance of UK doctors in training, as evaluated at ARCP. 
We observed considerable differences in ARCP outcomes 
between UK graduates and international medical graduates (who 
registered via the PLAB system). Some, but certainly not all, 
of the differences in ARCP performance may be explained by 
disproportionately high rates of membership examination failure, 
especially in psychiatry and general practice. The observed 
intergroup differences also seem to be partly mediated by sex, 
age, and years in UK practice. 

In terms of evidence relating to the validity of the PLAB 
examination, raising the pass mark for both parts of the test may 
reduce the disparity in performance observed between 
international medical graduates registering via the test and UK 
graduates. We also observed that linguistic performance is also 
strongly predictive of later ARCP performance. Moreover, 
superior performance on both parts of the PLAB test is 
independently and significantly predictive of better ARCP 
outcomes. These benefits seem to be partly mediated by 
interactions with other factors, such as length of exposure to 
UK practice. Time spent working in the UK is likely to help 
international medical graduates to become more confident in 
relation to the cultural aspects of healthcare, as well as 
enhancing clinical competencies. 

Comparison with other studies and possible 
interpretations 

Our findings are highly concordant with those of the 
accompanying report, studying the performance at MRCGP and 
MRCP of international medical graduates and UK graduates,' 8 
in that only the highest scoring PLAB candidates approached 
or reached equivalent levels of performance to UK graduates. 
This suggests that the problem of non-equivalence is present 
across most (although not all) medical specialties and is not 
fully explained by intergroup differences in postgraduate 
examination performance in medical specialties or general 
practice. These related studies provide strong evidence that the 
PLAB examination is, in its current form, unable to ensure 
equivalence between these two groups of doctors. 

After adjustment for rates of membership examination failure, 
the effect of being an international medical graduate changed 
most in general practice and psychiatry but relatively little in 
other specialty groups. Both these specialties have postgraduate 
examinations that have previously been shown to have had 
relatively high failure rates in international medical graduates 
compared with UK graduates. 14 25 The possible causes of these 
differential pass rates have been intensively debated over the 
past year or so, and these discussions are likely to continue in 
a legal arena. Both specialties contain above average proportions 
of international medical graduates (psychiatry is the specialty 
group with the highest proportion). This will allow any 
intergroup distinctions present to surface, as any analysis will 
be adequately powered to show such differences with a high 
level of confidence. Secondly, both specialties place a high 
degree of emphasis on communication. This is likely to be true 



not just in the pure linguistic sense but also in a cultural and 
subcultural context. For example, in psychiatry, a certain amount 
of judgment must be exercised to differentiate culturally (or 
subculturally) normal from pathological ideation. This situation, 
which may require more advanced communication skills, is 
likely to place international medical graduates at an overall 
disadvantage. Thirdly, both general practice and psychiatry are 
"shortage specialties," in which recruitment and retention of 
medical staff is recognised as challenging: a recent report 
highlighted that there are, on average, just 1.8 and 1.3 applicants 
to core training jobs in psychiatry and general practice, 
respectively, compared with 9.6 in the most competitive 
speciality of neurosurgery. 26 In practice, this may mean that 
PLAB candidates who passed with lower scores may be unable 
to secure a post in their preferred specialism but successfully 
apply for psychiatric or general practice training. This is likely 
to lead to higher rates of examination failure and less satisfactory 
outcomes at ARCP. 

As suggested by Esmail and Roberts, 14 racial bias in such 
examinations cannot be ruled out entirely, especially given that 
non-white UK graduates were noted to have performed more 
poorly on the clinical skills assessment component, even 
allowing for achievement on the knowledge based portion of 
the examination. This observation is consistent with the findings 
of an earlier systematic review of ethnicity and medical 
academic performance. 27 However, the evidence for 
candidate-examiner interactions relating to ethnic status in such 
examinations is fairly weak. 28 29 It is likely that more complex 
reasons underlie these disparities, including the "streaming" of 
more able candidates (both UK graduates and international 
medical graduates) into better deaneries and training posts, as 
well as more subtle interactions between cultural factors and 
examination ratings in observed clinical examinations and 
exercises, the latter of which contribute to ARCP outcomes. 
Our observation that international medical graduates were 
significantly less likely than UK graduates to have "out of 
programme experience" would be consistent with differences 
in the quality of training. Such a difference may reflect relatively 
reduced opportunities for involvement of international medical 
graduates in clinical research or their overrepresentation in 
specialties in which such training opportunities are less freely 
available. 

Strengths and potential limitations 

This analysis used a national sample that, in theory, included 
all doctors who had passed both parts of the PLAB during the 
timeframe of the study. The international medical graduates 
who passed the PLAB after 2010 but underwent ARCPs from 
2010 to 2012 would not have been included in this analysis. 
This is likely to have been a relatively small proportion of the 
international medical graduates who registered via the PLAB 
test and who underwent ARCPs during the study timeframe and 
is unlikely to have significantly affected the validity of our 
findings. Moreover, data on demographic factors such as sex, 
age, and years since UK registration were relatively complete. 
Data on ethnicity were less complete, although this was 
addressed through multiple imputation. The use of multiple 
imputation permitted the inclusion of observations in the models, 
including ethnicity, that would otherwise have been excluded 
due to missing data. This increases study power and potentially 
reduces the risk of bias while preserving the standard errors for 
the model estimates. 30 As ethnicity could be predicted from the 
other variables in the complete data, we were able to use a 
plausible model to impute the missing values. However, multiple 
imputation is based on the "missing at random" assumption 
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(that is, that the absent values are likewise related to the 
observed variables). Although the findings of our sensitivity 
analysis tend to suggest that ethnicity was missing at random 
(see technical appendix), we had no way of robustly testing this 
assumption. This represents the primary limitation of multiple 
imputation in such circumstances. 31 Moreover, little doubt exists 
that the analysis would have been more informative if a greater 
degree of detail were available for both reported ethnicity and 
specific country of qualification. In this case, the 
dichotomisation of ethnicity into non-white/white, to preserve 
anonymity, 32 meant that few conclusions could be drawn 
regarding the potential effect of this variable on ARCP 
performance; generally, this factor served as a proxy for 
international medical graduates status. Thus the validity of 
drawing comparisons regarding the effect of this ethnic status 
across both international medical graduates and UK graduates, 
in whom the meaning of the construct may be very different, is 
questionable. Further work should focus on finer grained 
analysis in which graduates from different countries are 
compared. Such analyses should discriminate between 
international medical graduates who were taught in English and 
those taught in other languages, given our findings in relation 
to IELTS scores. One should also bear in mind that the ARCP 
data were obtained from doctors who were further on in their 
training than foundation year two. Therefore, we could speculate 
that intervening factors could have affected the disparity in 
performance between PLAB graduates and UK graduates. For 
example, international medical graduates may have experienced 
a poorer quality of training. 

In comparison with royal college examinations, little is known 
about the reliability of the ARCP process. There may be 
instances of outliers, in terms of ARCP outcomes, in specific 
specialties within certain deaneries. 33 Moreover, there may have 
been some inconsistency in the reporting of the reason for an 
unsatisfactory ARCP outcome (such as examination failure) in 
2010 (Smith D. Reliability of ARCP data collection by the 
GMC, 2010-2012. Personal communication to P A Tiffin, 2014). 
Nevertheless, compared with examinations, ARCPs may be a 
closer, although still imperfect, proxy for clinical practice, as 
they consider a relatively wide range of evidence provided by 
a trainee. However, like clinical examinations, they cannot be 
considered free from cultural influences and opportunities for 
assessor bias, which would seem to be a priority area for future 
research. The extent to which international medical graduates 
are represented as ARCP panel members or what effect, if any, 
this may have on the outcomes of assessments is currently 
unknown. Certainly, monitoring the world region of qualification 
as well as the ethnicity of postgraduate assessors and examiners 
may be important in research relating to potential bias or 
discrimination. 

Although we were able to analyse the data according to specialty 
group, the low numbers of trainee doctors (and especially 
international medical graduates) in some of these (such as 
ophthalmology and public health) precluded adequately powered 
subgroup analysis in some cases. Moreover, the role that 
examinations play in ARCP outcome differs according to 
specialty; some have examinations earlier on in training, whereas 
others have "exit" fellowship examinations that allow entry to 
the specialist register. However, we hoped that variation across 
specialties would "wash out" so that an overall, average, picture 
of the performance of international medical graduates registering 
via the PLAB relative to UK graduates at ARCP could still be 
obtained. No data on deanery were available to the research 
team (this may have identified trainees in smaller specialties). 
However, deanery location has previously been shown to have 



a minimal effect on overall postgraduate performance; a recent 
report cited the intraclass correlation for deanery on MRCGP 
pass rates as generally less than 0.001. 14 

Understanding whether particular components of the PLAB 
examination, such as history taking, were especially predictive 
of ARCP performance would have been desirable. 
Unfortunately, this was not possible as PLAB subtest scores 
are not standardised between diets. Also, our analysis included 
only international medical graduates who obtained registration 
via the PLAB examination, so these findings cannot be applied 
to those who obtained GMC registration via alternative routes. 

Conclusions and implications for policy 

Although several demographic variables are predictive of ARCP 
performance, the main purpose of the PLAB examination is to 
ensure equivalence between international medical graduates 
and UK graduates at the end of foundation year one. However, 
these findings will contribute to continued discussions about 
the role that ethnicity plays in the assessment of UK doctors in 
training. Certainly, further detailed research and investigation 
is required before any assumption of systematic racial bias is 
made. The British Medical Association has recommended, 
following a survey of medical royal colleges, that data collection 
on the demographic diversity of their postgraduate examination 
candidates should be improved; only eight of 18 colleges 
reported analysing pass rates in relation to characteristics such 
as sex, ethnicity, disability, religious faith, and sexual 
orientation. 34 

Patients need reassurance that doctors qualifying from overseas 
are as competent and professional as UK graduates, irrespective 
of any underlying demographic differences. Language 
proficiency is considered to be an important component of a 
physician's ability and could be considered a necessary, if not 
sufficient, condition for clinical competence. 35 Nevertheless, 
the effect of IELTS scores on the disparity between international 
medical graduates' and UK graduates' ARCP performance was 
striking. The IELTS often uses examples of English that are 
quite technical in nature, as well as including role play in the 
speaking component. The test may thus be evaluating constructs 
relating to cognitive ability that are, to some extent, distinct 
from pure linguistic performance. This may partly explain the 
relatively strong ability of IELTS scores to predict ARCP 
performance in international medical graduates. Improving 
overseas doctors' confidence in culturally sensitive 
communication may be effective in improving consultation 
skills; one Scottish pilot of such a training programme reported 
encouraging findings. 36 Certainly, our results suggest that 
increasing the IELTS score (or equivalent) at which international 
medical graduates are permitted to take the PLAB examination 
will reduce the disparity in ARCP performance compared with 
UK graduates. There may also be ways in which health services 
can support overseas doctors in adapting to the culture of the 
UK. 37 

The pass mark for both parts of the PLAB would clearly need 
to be set much higher to substantially reduce the differences in 
performance observed between international medical graduates 
and UK graduates. In line with the findings from the parallel 
study, 18 our results suggest that the pass mark would have to be 
raised considerably (by at least 30 points for part 1 of the PLAB) 
to substantially move towards equivalence; so much so that very 
few of the current PLAB candidates would have passed. This 
does not mean that current PLAB graduates would not succeed 
at an examination with a higher pass mark, but they would need 
to raise their performance to do so. An alternative would be to 
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introduce a different examination. Our findings also indicate 
that limiting the number of times the PLAB examinations can 
be taken is likely to result in some closing of the performance 
gap between international medical graduates and UK graduates. 
Together, such changes would create severe workforce planning 
challenges for the NHS, which has traditionally relied on 
international medical graduates, especially in the less popular 
specialties such as psychiatry. It is also interesting to note that 
in Canada international medical graduates must pass both parts 
of the Medical Council of Canada examination — the Medical 
Council of Canada Qualifying Examination (MCCQE). The 
pass mark is identical for international medical graduates and 
Canadian medical graduates. 38 Despite this, as in the UK, 
international medical graduates later show higher failure rates 
at postgraduate examination. i9 Thus, it may be erroneous to 
conclude that if only the pass mark of the PLAB were set high 
enough, international medical graduates in the UK would show 
identical performance at later assessments. Even marked 
variation between graduates of different UK medical schools 
in performance at royal college membership examinations has 
been previously reported. 4 " As such, achieving early 
"equivalence" between UK graduates and international medical 
graduates may be unfeasible and partly depend on exposure to 
UK practice and culture. Rather, medical regulators must decide 
on the levels of clinical skill and professionalism required for 
UK practice and implement policy accordingly. 16 In the 
meantime, we would recommend that the IELTS (or equivalent) 
score required to take the PLAB is increased and that pass marks 
for both parts of the PLAB test are raised, with the number of 
resits permitted restricted. These steps are likely to result in 
reduced disparities in postgraduate performance between PLAB 
graduates and UK graduates. 
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What is already known on this topic 

Compared with UK medical graduates, international medical graduates are more likely to fail postgraduate royal college examinations 

International medical graduates are also more likely to be referred to the General Medical Council for concerns relating to fitness to 
practise 

However, the extent to which the Professional and Linguistic Assessments Board (PLAB) test ensures that international medical graduates 
show equivalent performance to UK graduates who have completed foundation year one is unclear 

What this study adds 

The PLAB examination system does not result in equivalent performance between international medical graduates and UK graduates, 
as evaluated later at Annual Review of Competence Progression (ARCP) 

Performance at both the International English Language Test System and the two parts of the PLAB are significantly predictive of ARCP 
outcome in international medical graduates 

Some, but not all, of the differences in ARCP performance may be explained by disproportionately high rates of membership examination 
failure, especially in psychiatry and general practice 
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Tables 



Table 1 1 List of possible outcomes awarded to medical trainees at Annual Review of Competence Progression and Record of In-Training 
Assessment 


Outcome 


Description 


Annual Review of Competence Progression 


1 Satisfactory progress; competences achieved as expected 


2 


May progress but requires specific/targeted training to achieve certain competences 


3 


Has not achieved competences required to progress; up to 12 months' (6 months in Core and GP) additional training required 


4 


Released from training programme with or without specified competences 


5 


Incomplete evidence provided 


6 


Recommendation for completion of training having gained all required competences 


7 


Fixed term specialty outcome — specific competences achieved 


7.1 


Satisfactory progress in or completion of Locum Appointment for Training (l_AT)/Fixed Term Speciality Training Appointments (FTSTA) 
placement 


7.2 


Development of specific competences required; additional training time not required — LAT/FTSTA placement 


7.3 


Inadequate progress by trainee; additional training time required — LAT/FTSTA placement 


7.4 


Incomplete evidence presented — LAT/FTSTA placement 


8 Out of programme experience; time may or may not count towards training 


Record of In-Training Assessment 


C 


Satisfactory progress; competences achieved as expected 


D 


May progress but requires specific/targeted training to achieve certain competences 


E 


Has not achieved competences required to progress; up to 12 months' (6 months in Core and GP) additional training required 


F 


Out of programme experience; time may or may not count towards training — usually refers to period of research or specialist training 
undertaken, either in UK or abroad 


G 


Recommendation for completion of training having gained all required competencies 
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able 2| Demographic characteristics of 53 436 doctors with Annual Review of Competence Progression (ARCP) outcomes available. Values 
are numbers (percentages) unless stated otherwise 



Characteristic 


UK graduates (n=42 017) 


International medical graduates (n=1 1 
419) 


Proportion missing 


Male sex (%) 


18 666 (44) 


6877 (60) 


0 


Mean (SD) age, years 


31.61 (4.1) 


35.89 (4.1) 


7/53 436 (0.000) 


Mean (SD) years of UK experience 


6.69 (3.5) 


6.12 (1.9) 


0 


Non-white ethnicity 


9381/32 458 (29) 


8387/8894 (94) 


12 084/53 436 (23) 


Median (interquartile range) No of ARCPs 
taken 


2(1-3) 


2 (2-3) 


0 



All intergroup differences are significant at P<0.01 level. P values for intergroup testing were derived from x z test in case of sex, ethnicity, and number of ARCPs 
taken. Kruskall-Wallis test was used to compare mean age and years of UK based experience at ARCP between groups. 
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Table | Proportion of international medical graduates registering via PLAB test in each speciality group along with proportions of Annual 
Reviews of Competence Progression (ARCP) in which outcome indicated satisfactory progress. Values are numbers (percentages) 



Specialty 


Proportion of international medical graduates Proportion satisfactory ARCP outcomes 


Anaesthetics and intensive care 


708/5371 (13.2) 


10 131/12 680 (79.9) 


Emergency medicine 


258/1723 (15.0) 


2482 /3422 (72.5) 


General practice 


3327/14 564 (22.8) 


23 287/28 997 (80.3) 


Medicine 


2663/13 907 (19.2) 


26 894/37 292 (72.1) 


Obstetrics and gynaecology 


861/2456 (35.1) 


4232/6008 (70.4) 


Ophthalmology 


88/758 (1 1 .6) 


1443/1819 (79.3) 


Paediatrics 


1128/4146 (27.2) 


7444 /1 0 210 (72.9) 


Pathology and clinical sciences 


237/1058 (22.4) 


1890/2385 (79.3) 


Psychiatry 


1738/3886 (44.7) 


6128/8535 (71.8) 


Public health 


24/288 (8.3) 


529/605 (87.4) 


Radiology 


264/1412 (18.7) 


2766/3220 (85.9) 


Surgery 


885/7402 (12.0) 


13 653/17 613(77.5) 
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Table 4| Random effects ordinal logistic regression model for obtaining less satisfactory Annual Review of Competence Progression 
(ARCP) outcome for international medical graduates (IMG) versus UK graduates, adjusted for sex, age, years of UK based experience, and 
ethnicity* 



Predictor 


Odds ratio (95% CI) 


Male sex 


1 .29 (1 .23 to 1 .34) 


Age 


1.00t (0.98 to 1.01) 


UK experience 


0.71 (0.68 to 0.74) 


Non-white ethnicity 


0.72:f: (0.49 to 1 .06) 


Age/UK experience interaction 


1.01 (1.00 to 1.01) 


IMG status/non-white ethnicity interaction 


0.69 (0.54 to 0.89) 


Age/non-white ethnicity interaction 


1.02(1.01 to 1.03) 


IMG status 


1.63(1.30 to 2.06) 


*Data from 119 352 ARCPs undergone by 52 871 doctors were used, with outcomes associated with examination failure excluded. All odds ratios are significant 

at P<0.01 level unless otherwise indicated. 

tP=0.39. 

$P=0.1. 
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able 5| Random effects ordinal logistic regression model for obtaining less satisfactory Annual Review of Competence Progression 
(ARCP) outcome (n=24 474) as predicted by Professional and Linguistic Assessments Board (PLAB) examination scores at first attempt 



in international medical graduates (n=10 678)* 


Predictor 


Odds ratio (95% CI) 


PLAB part 1 


Male sex 


1.24 (1.14 to 1.36) 


Age 


1.04 (1.03 to 1.05) 


UK experience 


0.91 (0.88 to 0.93) 


Overall IELTS score 


0.77 (0.70 to 0.84) 


PLAB part 1 /experience interaction 


0.99 (0.99 to 0.99) 


PLAB part 1 score 


1.00t (0.99 to 1.01) 


PLAB part 2 


Male sex 


1.16 (1.07 to 1.27) 


Age 


1.04(1.03 to 1.05) 


UK experience 


0.89 (0.87 to 0.92) 


Overall IELTS score 


0.80 (0.72 to 0.87) 


PLAB part 2 score 


0.95 (0.95 to 0.96) 


IELTS=lnternational English Language Test System. 

*ARCP outcomes associated with examination failure were excluded. All odds ratios are significant at P<0.01 level unless otherwise indicated. 
tP=0.69. 
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Table £ | Random effects ordinal logistic regression model estimating effect of number of attempts at Professional and Linguistic Assessments 
Board (PLAB)examination, parts 1 and 2, on obtaining less satisfactory Annual Review of Competence Progression (ARCP) outcomes 
(n=24 474) in international medical graduates (n=10 678)* 



Predictor 


Odds ratio (95% CI) 


Part 1 of PLAB test 


Male sex 


1.24 (1.1 4 to 1.35) 


Age 


1.04 (1.03 to 1.05) 


UK experience 


0.89 (0.87 to .92) 


Overall IELTS score 


0.73 (0.67 to 0.80) 


Two attempts at PLAB part 1 


1.30 (1.1 6 to 1.45) 


Three attempts at PLAB part 1 


1.69 (1.41 to 2.02) 


Four or more attempts at PLAB part 1 


1.60 (1.29 to 1.98) 


Part 2 of PLAB test 


Male sex 


1.21 (1.11 to 1.32) 


Age 


1.04(1.03 to 1.05) 


UK experience 


0.89 (0.86 to 0.91) 


Overall IELTS score 


0.73 (0.67 to 0.80) 


Two attempts at PLAB part 2 


1.34 (1.20 to 1.50) 


Three or more attempts at PLAB part 2 


1.49f (1.14 to 1.94) 


IELTS=lnternational English Language Test System. 

'Passing the exam at first sitting was used as base (comparator) category. ARCP outcomes associated with exam failure were excluded. All odds ratios are 

significant at P<0.001 level unless otherwise indicated. 

tP=0.003. 
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Figures 



148143 ARCPs related to 60654 doctors who completed at least one ARCP in the period 2010-2012 






6473 ARCPs relating to 


17 ARCPs excluded as they were related to 5 
doctors listed as UKGs hut nad sat at least one 
part of the PLAB 


2700 EU graduates excluded 




8850 ARCPs relating to 3827 IMGs who did 




not complete the PLAB were excluded 


132803 ARCPs relating to 54122 doctors who were either IMG PLAB graduates or UK graduates 







7595 ARCPs not related to performance 
(e g out of programme experience) 
(686 doctors had only these ARCP outcomes 
recorded and were excluded) 




125208 ARCPs relating to performance in 53436 doctors who were either IMG PLAB graduates or UK graduates 



Complete data on predictor variables (excluding ethnicity) 
on 53429 doctors associated with 125193 ARCPs. 5841 
of which were associated with exam failure 



Complete or imputed data on predictor variables including 
ethnicity on 53429 doctors associated with 1 251 93 ARCPs 
(ethnicity available for 4 1 347 doctors and imputed in 
12082 cases in this final dataset) 



Fig 1 Flow of data through study 




l Good progress 2: Incomplete evident* J: Targeted training r'ed 4: Training extended 5: Release from programme 

ARCP Outcome Category 



Fig 2 Annual Review of Competence Progression (ARCP) outcomes (n=125 208) for doctors in sample (n=53 436). 
International medical graduates registered via Professional and Linguistic Assessments Board examination (PLAB IMG) 
and UK graduates (UKG) shown separately 
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1 Good process 2: Incomplete evidence J Ta«je ted (raining r'ed 4 Tranmgevtended 5 Release from programme 



Fig 3 Annual Review of Competence Progression (ARCP) outcomes in sample (n=1 1 9 367) not associated with postgraduate 
examination failure in sample of doctors (n=52 878). International medical graduates registered via Professional and 
Linguistic Assessments Board examination (PLAB IMG) and UK graduates (UKG) shown separately 



3 4 5 6 7 8 9 
Medical speciality group ranked by size of odds ratio 



10 



11 



Odds ratios and standard errors- All ARCP outcomes 
Odds ratios and standard errors- Exam failures excluded 



Fig 4 Results from random effects ordinal regression models for obtaining less satisfactory Annual Review of Competence 
Progression (ARCP) outcome as international medical Professional and Linguistic Assessments Board medical graduate 
(versus UK graduate) by medical specialty. Estimates depicted in grey are from analyses in which ARCP outcomes associated 
with royal medical college membership examination failure were excluded 
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Overall IELTS score at 'pass' 

Fig 5 Unadjusted (raw) odds ratios for risk that Annual Review of Competence Progression (ARCP) outcome falls into less 
satisfactory category (international medical graduate registered via Professional and Linguistic Assessments Board versus 
UK graduate) for different simulated International English Language Test System (IELTS) "pass" score thresholds. Odds 
ratios and associated standard errors were derived from a series of multilevel ordinal logistic regression models 
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Duodecile at PLAB part 1 pass, relative to pass mark 

Fig 6 Unadjusted (raw) odds ratios for risk that Annual Review of Competence Progression (ARCP) outcome falls into less 
satisfactory category (international medical graduate registered via Professional and Linguistic Assessments Board (PLAB) 
versus UK graduate) for different simulated PLAB part 1 pass marks. Odds ratios and associated standard errors were 
derived from a series of multilevel ordinal logistic regression models. Twelfths are used to rank international medical 
graduates' performances on part 1 of the examination, with those in the first group having a score at least 32 points above 
pass mark and those in the 1 2th group being in the lowest scoring twelfth (score of at least 0 above pass) of PLAB graduates 
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Duodecile at PLAB part 2 at pass, relative to pass mark 
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Fig 7 Unadjusted (raw) odds ratios for risk that Annual Review of Competence Progression (ARCP) outcome falls into less 
satisfactory category (international medical graduate registered via Professional and Linguistic Assessments Board (PLAB) 
versus UK graduate) for different simulated PLAB part 2 pass marks. Odds ratios and associated standard errors were 
derived from a series of multilevel ordinal logistic regression models. Twelfths are used to rank international medical 
graduates' performances on part 2 of the examination, with those in the first group having a score of at least 12.1 points 
above pass mark and those in the 12th group being in lowest scoring twelfth of PLAB graduates 
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